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Motivation 



An Alternate Approach to VM Construction 


® Most productions VMs are written by hand in C/C++ 

® VMs are very tedious and costly to write 
® Particularly for complex dynamically typed languages 
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Domain: Language Virtual Machines 


® Important goal of VM construction is performance 
® A good JIT is needed 

® Particularly for dynamically typed languages, where nothing is 
known statically 
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Problem: VMs have Monolithic Architecture 


o JITs are complex engineering artifacts 

® Architecture is often very complicated, with different concerns 
tangled up 

® Changing the language is a lot of effort 

9 Very hard to share infrastructure between VMs for different 
languages 
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Problem: VMs have Monolithic Architecture 


9 JITs are complex engineering artifacts 

9 Architecture is often very complicated, with different concerns 
tangled up 

9 Changing the language is a lot of effort 
9 Very hard to share infrastructure between VMs for different 
languages 
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Approach: Separation of Concerns 


Separate the following VM implementation concerns: 
® Language semantics 
® GenericJIT compilation issues 
® Generic optimizations 
® Language-specific optimizations 
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The RPython Project and PyPy 


® RPython is a language to implement interpreters 
® Interpreters are translated into C-based VMs 
® Various extra features are added, e.g. GC 
® Most mature interpreter is PyPy: Python in RPython 
® Long-running project, many contributors 
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A Meta-Tracing J IT for RPython 


® Apply the meta-tracing approach to RPython 
® Insert meta JIT into the generated VM 

® Contains generic JIT infrastructure: backends, integration, GC 
® Needs some hints from the interpreter author 
® Hints specify the main dispatch loop and the program counter 
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A Meta-Tracing J IT for RPython 


® Apply the meta-tracing approach to RPython 
® Insert meta JIT into the generated VM 

® Contains generic JIT infrastructure: backends, integration, GC 
® Needs some hints from the interpreter author 
® Hints specify the main dispatch loop and the program counter 
o Typical optimizations, plus some specific ones 


Interpreter in RPython 

MetaJIT Optim. 
Runtime 
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Meta-Tracing 



Tracing JITs 


® Dream: add a simple JIT component to an interpreter 
® Starts out interpreting and focuses on loops. 

® Tracer records activity of interpreter for important loops. 
® Conditions are turned into guards. 
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Tracing JITs 


9 Dream: add a simple JIT component to an interpreter 
9 Starts out interpreting and focuses on loops. 

9 Tracer records activity of interpreter for important. 

9 Conditions are turned into guards. 
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Tracing JITs 


User program (lang FL) 

if x < 0 : 
x = x + 1 
else : 

x = x + 2 
x = x + 3 
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Tracing JITs 


User program (lang FL) 

Trace when x is set to 6 

if x < 0 : 

guard_type (x, int) 

x = x + 1 

guard_type ( 0 , int) 

else : 

guard_not_less_than (x, 0) 

x = x + 2 

guard_type (x, int) 

CO 

+ 

X 

II 

X 

guard_type (2, int) 
x = int_add(x, 2) 
guard_type (x, int) 
guard_type (3, int) 
x = int_add(x, 3) 
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Tracing JITs 


User program (lang FL) 

Optimised trace 

if x < 0 : 
x = x + 1 
else : 

x = x + 2 
x = x + 3 

guard_type (x, int) 
guard_not_less_than (x, 0) 

x = int_add(x, 5) 
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Guards 


® Conditions are turned into guards. 

® They check that the same control flow is followed. 

® When they fail, go back to interpretation. 

® Side traces are attached to commonly failing guards. 

® Tracing works best if subsequent iterations of a loop follow the 
same control flow. 
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State Transitions TracingJIT 
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In Practice 







Meta-Tracing 


® Solution: Trace the interpreter, not the program. 

® Interpreters are big loops with complex control flow. 
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Adding a JIT to an RPython interpreter 


pc = 0 

while 1 : 

instr = load_next_instruction (pc) 
if instr == POP : 
stack . pop ( ) 
pc += 1 

elif instr == BRANCH: 

off = load_branch_jump (pc) 

pc += off 

elif . . . : 

Observation: interpreters are big loops. 
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Adding a JIT to an RPython interpreter 


pc = 0 

while 1 : 

j it_merge_point (pc) 
instr = load_next_instruction (pc) 
if instr == POP : 
stack . pop ( ) 
pc += 1 

elif instr == BRANCH: 

off = load_branch_jump (pc) 
if off < 0: can_enter_j it (pc) 

pc += off 

elif . . . : 

Observation: interpreters are big loops. 
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Meta-Tracing 


® A lot of the interpreter data structure manipulations are 
optimized away. 

9 Examples: Stack manipulation etc. 

9 Some technical challenges, but separation works well. 
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Generic Optimizations 


a Typical compiler optimizations 
® Easy to implement, because of traces 
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Generic Optimizations 


® Typical compiler optimizations 
® Easy to implement, because of traces 
® Interesting new one: allocation removal 

® Dynamic languages allocate a lot of objects, e.g. for primitive 
boxes 

o Objects often have limited predetermined lifetime 
® a + b * c 

® Remove intermediate allocations in traces 


Software Development Team 


23/38 


http://soft-dev.org/ 




Runtime Feedback 



Language-Specific Runtime Feedback 


o JIT gets its power by observing the running program 
9 Bare meta-tracing does not know any details of the language 
implemented 

9 Language implementer can provide extra information with 
more hints 

9 These often express how the language is typically used 
9 Such information is only implicit or absent from the interpreter 
source 
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Runtime 
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Example: Language-Specific Runtime Feedback 


At a method callsite, the called method is often always the same: 

def lookup (els, methname) : 

def send(obj, messagename, arguments): 
els = ob j . getclass ( ) 

meth = lookup (els, messagename) 
return meth . call (obj , arguments) 
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Example: Language-Specific Runtime Feedback 


At a method callsite, the called method is often always the same: 

def lookup (els, methname) : 


def send(obj, messagename, arguments): 
els = ob j . getclass ( ) 
promote (els) 

meth = lookup (els, messagename) 
return meth . call (obj , arguments) 
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Example: Language-Specific Runtime Feedback 


At a method callsite, the called method is often always the same: 

@elidable 

def lookup (els, methname) : 


def send(obj, messagename, arguments): 
els = ob j . getclass ( ) 
promote (els) 

meth = lookup (els, messagename) 
return meth . call (obj , arguments) 
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Case Studies 



PyPy, an RPython VM for Python 



® How the RPython project got started. 

® Very compatible and fast implementation of Python 2.7. 
® Big developer community with various interests. 

® Around 60K LoC (interpreter) and 190K LoC (modules) of 
RPython code. 
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Hints in PyPy 


® About 400 hints in the interpreter. 

® Continuous process to add more. 

® Mostly concerned with core features of the language, 
a Global lookups 
a Method calls 
a Attribute reads 

a Data structures (e.g. lists and sets) 
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Example: Attribute Reads in Python 


What happens when an attribute x . m is read? (simplified) 

® check for the presence of x. getattribute , if there, call 

it 

9 look for the name of the attribute in the object's dictionary, if it's 
there, return it 

9 walk up the MRO of the object and look in each class' dictionary 
for the attribute 

9 if the attribute is found, call its get attribute and return 

the result 

9 if the attribute is not found, look for x. getattr , if there, 

call it 

9 raise an AttributeError 
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Python Benchmarks 


Speedup over CPython 




1.64 




1 




1.02 





0.4 



CPython 

Psyco 

PyPy-interp PyPy-meta 


7.52 


PyPy-full 


Software Development Team 


33/38 


http://soft-dev.org/ 





Shootout Benchmarks 


Speedup Over PyPy-full 
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PyPy-full 
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Demo 
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Pyrolog 


® A Prolog interpreter in RPython 
® Very different execution model than Python 
® Still decent performance improvements 
® About 1 5K LoC of RPython code 
® 70 hints 
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Prolog Benchmarks 
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Speedup over Sicstus Interp 
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Summary 


® Using the RPythonJIT, most of the J IT infrastructure is shared 
between languages 

9 Only an interpreter and some hints are needed 
9 Can support languages as different as Prolog and Python 

9 Other languages: PHP, Ruby, Smalltalk, Racket, SQLite, CPU 
emulators, ... 

9 "Gives you 80% of a great JIT 20% of the effort" 
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